Method for analyzing an image of a dental arch

ABSTRACT

A method for enriching a learning base intended for the training of a deep learning device. Production of an updated reference model of a dental arch and segmentation to produce a tooth model for each tooth. Acquisition of at least one updated image of the arch. For each updated image, search for virtual acquisition conditions suitable for an acquisition of a reference image of the updated reference model exhibiting a maximum match with the updated image, and acquisition of the reference image. Identification, in the reference image, of at least one reference tooth zone representing a tooth model by comparison of the updated image and the reference image determination of an updated tooth zone representing the tooth model. Assignment, to the updated tooth zone, of the tooth attribute value or values of the tooth model. Addition of the updated image enriched with value or values in the learning base.

TECHNICAL FIELD

The present invention relates to the field of dental arch imageanalysis.

STATE OF THE ART

The most recent orthodontic treatments use images to assess thetherapeutic situations. This assessment is conventionally performed byan orthodontist, which requires the patient to transmit these images tothe orthodontist, even to make an appointment.

There is an ongoing need for a method simplifying the analysis of theimages of dental arches of patients.

One aim of the invention is to address this need.

SUMMARY OF THE INVENTION

The invention proposes a method for analyzing an image, called “analysisimage”, of a dental arch of a patient, a method in which the analysisimage is submitted to a deep learning device, preferably a neuralnetwork, in order to determine at least one value of a tooth attributerelating to a tooth represented on the analysis image, and/or at leastone value of an image attribute relating to the analysis image.

Analysis by Tooth

The invention proposes in particular a method for detailed analysis ofan image called “analysis image” of a dental arch of a patient, saidmethod comprising the following steps:

-   -   1) creation of a learning base comprising more than 1000 images        of dental arches, or “historical images”, each historical image        comprising one or more zones each representing a tooth, or        “historical tooth zones”, to each of which, for at least one        tooth attribute, a tooth attribute value is assigned;    -   2) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3) submission of the analysis image to said at least one deep        learning device for it to determine at least one probability        relating to an attribute value of at least one tooth represented        on a zone representing, at least partially, said tooth in the        analysis image, or “analysis tooth zone”;    -   4) determination, as a function of said probability, of the        presence of a tooth of said arch at a position represented by        said analysis tooth zone, and of the attribute value of said        tooth.

A first deep learning device, preferably a neural network, may inparticular be implemented to assess a probability relating to thepresence, at a location of said analysis image, of an analysis toothzone.

A second deep learning device, preferably a neural network, may inparticular be implemented to assess a probability relating to the typeof tooth represented in an analysis tooth zone.

As will be seen in more detail hereinafter in the description, adetailed analysis method according to the invention advantageously makesit possible to immediately recognize the content of the analysis image.

The analysis image may advantageously be classified automatically. Itmay also be used immediately by a computer program.

The invention relies on the use of a deep learning device, preferably aneural network, the performance of which is directly linked to therichness of the learning base. There is therefore also a need for amethod that makes it possible to rapidly enrich the learning base.

The invention therefore relates also to a method for enriching alearning base, notably intended for the implementation of a detailedanalysis method according to the invention, said enrichment methodcomprising the following steps:

-   -   A) at an “updated” instant, production of a model of a dental        arch of a patient, or “updated reference model”, and        segmentation of the updated reference model so as to produce,        for each tooth, a “tooth model”, and, for at least one tooth        attribute, assignment of a tooth attribute value to each tooth        model:    -   B) preferably less than 6 months, preferably less than 2 months,        preferably less than 1 month, preferably less than a fortnight,        preferably less than 1 week, preferably less than 1 day before        or after the updated instant, preferably substantially at the        updated instant, acquisition of at least one, preferably at        least three, preferably at least ten, preferably at least one        hundred images of said arch, or “updated images” in respective        real acquisition conditions;    -   C) for each updated image, search for virtual acquisition        conditions suitable for an acquisition of an image of the        updated reference model, called “reference image”, exhibiting a        maximum match with the updated image in said virtual acquisition        conditions, and acquisition of said reference image;    -   D) identification, in the reference image, of at least one zone        representing a tooth model, or “reference tooth zone”, and, by        comparison of the updated image and of the reference image,        determination, in the updated image, of a zone representing said        tooth model, or “updated tooth zone”;    -   E) assignment, to said updated tooth zone, of the tooth        attribute value or values of said tooth model;    -   F) addition of the updated image enriched with a description of        said updated tooth zone and its tooth attribute value or values,        or “historical image”, in the learning base.

In particular, each execution of the method described in WO 2016/066651preferably generates more than three, more than ten, preferably morethan one hundred updated images which, by an automated processing bymeans of the updated reference model, may produce as many historicalimages.

In a particular embodiment, the method for enriching a learning basecomprises, in place of the steps A) to C), the following steps:

-   -   A′) at an initial instant, production of a model of a dental        arch of a patient, or “initial reference model”, and        segmentation of the initial reference model so as to produce,        for each tooth, a “tooth model”, and, for at least one tooth        attribute, assignment of a tooth attribute value to each tooth        model;    -   B′) at an updated instant, for example spaced apart by more than        a fortnight, preferably more than a month, even more than two        months from the initial instant, acquisition of at least one,        preferably at least three, preferably at least ten, preferably        at least one hundred images of said arch, or “updated images”,        in respective real acquisition conditions;    -   C′) for each updated image, search, by deformation of the        initial reference model, for an updated reference model and        virtual acquisition conditions suitable for an acquisition of an        image of the updated reference model, called “reference image”,        exhibiting a maximum match with the updated image in said        virtual acquisition conditions.

This method advantageously makes it possible, after generation of theinitial reference model, preferably by means of a scanner, to enrich thelearning base at different updated instants, without it being necessaryto perform a new scan, and therefore without the patient having to go tothe orthodontist. He or she may in fact acquire the updated images himor herself, as described in WO 2016/066651.

A single orthodontic treatment may thus lead to the production ofhundreds of historical images.

The invention relates also to a method for training a deep learningdevice, preferably a neural network, comprising an enrichment of alearning base according to the invention, then the use of said learningbase to train the deep learning device.

Global Analysis

The detailed analysis method described above advantageously allows for afine analysis of the analysis image, the situation of each beingpreferably assessed.

Alternatively, the deep learning device may be used globally, thelearning base containing historical images whose description provides aglobal attribute value for the image. In other words, the value of theimage attribute relates to the whole image and not to part of the image.The attribute is not then a. “tooth” attribute, but is an “image”attribute. For example, this image attribute may define whether, inlight of the image as a whole or of a part of the image, the dentalsituation “is pathological” or “is not pathological”, without each toothbeing examined. The image attribute also makes it possible to detect,for example, whether the mouth is open or closed, or, more generally,whether the image is suitable for a subsequent treatment, for examplewhether it makes it possible to monitor the occlusion.

The image attribute may in particular relate to

-   -   a position and/or an orientation and/or a calibration of an        acquisition apparatus used to acquire said analysis image,        and/or    -   a quality of the analysis image, and in particular relating to        the brightness, to the contrast or to the sharpness of the        analysis image, and/or    -   the content of the analysis image, for example to the        representation of the arches, of the tongue, of the mouth, of        the lips, of the jaws, of the gums, of one or more teeth or of a        dental, preferably orthodontic, appliance.

When the image attribute refers to the content of the image, thedescription of the historical images of the learning base specifies acharacteristic of this content. For example, it may specify the positionof the tongue (for example “retracted”) or the opening of the mouth ofthe patient (for example mouth open or closed) or the presence of arepresentation of a dental, preferably orthodontic, appliance and/or ofits condition (for example appliance intact, broken or damaged).

A tooth attribute value may be used to define a value for an imageattribute. For example, if a value of a tooth attribute is “toothdecayed”, the value of an image attribute may be “unsatisfactory dentalsituation”.

The image attribute may in particular relate to a therapeutic situation.

The invention proposes a method for global analysis of an analysis imageof a dental arch of a patient, said method comprising the followingsteps:

-   -   1′) creation of a learning base comprising more than 1000 images        of dental arches, or “historical images”, each historical image        comprising an attribute value for at least one image attribute,        or “image attribute value”;    -   2′) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3′) submission of the analysis image to the deep learning device        for it to determine, for said analysis image, at least one        probability relating to said image attribute value, and        determination, as a function of said probability, of a value for        said image attribute for the analysis image.

The image attribute may in particular relate to the orientation of theacquisition apparatus upon the acquisition of the analysis image. It mayfor example take the values “front photo”, “left photo” and “rightphoto”.

The image attribute may also relate to the quality of the image. It mayfor example take the values “insufficient contrast” and “acceptablecontrast”.

The image attribute may also relate to the dental situation of thepatient, for example relate to the presence of decay or to the conditionof a dental, preferably orthodontic, appliance worn by the patient(“degraded” or “in good condition” for example) or to the suitability ofthe dental, preferably orthodontic, appliance to the treatment of thepatient (for example “unsuitable” or “suitable”).

The image attribute may also relate to the “presence” or “absence” of adental, preferably orthodontic, appliance, or to the state of opening ofthe mouth (“mouth open”, “mouth closed” for example).

As will be seen in more detail hereinafter in the description, a globalimage analysis method according to the invention advantageously makes itpossible to immediately and globally assess the content of the analysisimage. In particular, it is possible to globally assess a dentalsituation and, for example, deduce therefrom the need to consult anorthodontist.

Definitions

A “patient” is a person for whom a method according to the invention isimplemented, independently of whether or not this person is following anorthodontic treatment.

“Orthodontist” should be understood to mean any person qualified toprovide dental care services, which also includes a dentist.

A “dental part”, in particular “orthodontic part”, should he understoodto mean all or part of a dental, in particular orthodontic, appliance.

An orthodontic part may in particular be an orthodontic aligner. Such analigner extends in such a way as to follow the successive teeth of thearch on which it is fixed. It defines a generally “U” shaped channel,the shape of which is determined to ensure the fixing of the aligneronto the teeth, but also as a function of a desired target positioningfor the teeth. More specifically, the shape is determined in such a waythat, when the aligner is in its service position, it exerts stressestending to move the teeth being treated to their target positioning, orto hold the teeth in this target positioning.

The “service position” is the position in which the dental ororthodontic part is worn by the patient.

“Model” should be understood to be a digital three-dimensional model. Anarrangement of tooth models is therefore a model.

“Image” should be understood to be an image in two dimensions, like aphotograph or an image taken from a film. An image is formed by pixels.

A “reference image” is a view of a “reference” model.

“Image of an arch” or “model of an arch” should be understood to be arepresentation of all or part of said arch. Such a representation ispreferably in colors.

The “acquisition conditions” of an image specify position and theorientation in space of an acquisition apparatus acquiring this image inrelation to the teeth of the patient (real acquisition conditions) or toa model of teeth of the patient (virtual acquisition conditions), andpreferably the calibration of this acquisition apparatus. Acquisitionconditions are said to be “virtual” when they correspond to a simulationin which the acquisition apparatus would be in said acquisitionconditions (theoretical positioning and preferably calibration of theacquisition apparatus) relative to a model.

In virtual conditions of acquisition of a reference image, theacquisition apparatus may be also qualified as “virtual”. The referenceimage is in fact acquired by a hypothetical acquisition apparatus,having the characteristics of the “real” acquisition apparatus used forthe acquisition of the real images, and in particular of the updatedimages.

The “calibration” of an acquisition apparatus is made up of all thevalues of the calibration parameters. A “calibration parameter” is aparameter intrinsic to the acquisition apparatus (unlike its positionand orientation) whose value influences the image acquired. Preferably,the calibration parameters are chosen from the group formed by thediaphragm aperture, the exposure time, the focal distance and thesensitivity.

A “discriminating information item” is a characteristic feature whichmay be extracted from an image (“image feature”), conventionally bycomputer processing of this image.

A discriminating information item may exhibit a variable number ofvalues. For example, a contour feature may be equal to 1 or 0 dependingon whether a pixel belongs or does not belong to a contour. A brightnessfeature may take a large number of values. The processing of the imagemakes it possible to extract and quantify the discriminating informationitem.

The discriminating information item may be represented in the form of a“map”. A map is thus the result of a processing of an image in order toreveal the discriminating information item, for example the contour ofthe teeth and of the gums.

“Match” or “fit” between two objects designates a measurement of thedifference between these two objects. A match is maximal (“best fit”)when it results from an optimization making it possible to minimize saiddifference.

An object modified to obtain a maximum match may be qualified as“optimal” object.

Two images or “views” which exhibit a maximal match representsubstantially at least one same tooth, in the same way. In other words,the representations of the tooth on these two images may besubstantially superimposed.

The search for a reference image exhibiting a maximal match with anupdated image is performed by searching for the virtual acquisitionconditions of the reference image exhibiting a maximal match with thereal acquisition conditions of the updated image.

By extension, a model exhibits a maximal match with an image when thismodel has been chosen from several models because it allows a viewexhibiting a maximal match with said image and/or when this image hasbeen chosen from several images because it exhibits a maximal match witha view of said model.

In particular, an updated image is in maximal match with a referencemodel when a view of this reference model provides a reference imageshowing a maximal match with the updated image.

The comparison between two images results preferably from the comparisonof two corresponding maps. “Distance” is used conventionally to denote ameasurement of the difference between two maps or between two images.

The “metaheuristic” methods are known optimization methods. They arepreferably chosen from the group formed by

-   -   evolutionary algorithms, preferably chosen from: evolution        strategies, genetic algorithms, differential evolution        algorithms, distribution estimation algorithms, artificial        immunity systems, shuffled complex evolution path recomposition,        simulated annealing, ant colony algorithms, particular swarm        optimization algorithms, search with taboos, and the GRASP        method;    -   the kangaroo algorithm,    -   the Fletcher and Powell method,    -   the sound effects method,    -   stochastic tunneling,    -   random-restart hill climbing,    -   the cross-entropy method, and    -   the hybrid methods between the abovementioned metaheuristic        methods.

“Description” of an image denotes information particularly relating tothe definition of the tooth zones of this image and to the toothattribute values which are associated with them, and/or relating to animage attribute value of said image. The number of possible values for atooth attribute or an image attribute is unlimited.

A “historical” image is an image of a dental arch enriched with adescription. The tooth zones of a historical image are qualified as“historical tooth zones”.

“Comprising” or “including” or “exhibiting” should be interpretedwithout restriction, unless indicated otherwise.

BRIEF DESCRIPTION OF THE FIGURES

Other features and advantages of the invention will become more apparenton reading the following detailed description and studying the attacheddrawing in which:

FIG. 1 represents, schematically, the different steps of a method fordetailed analysis of an image, according to the invention;

FIG. 2 represents, schematically, the different steps of a method forenriching a learning base, according to the invention;

FIG. 3 represents, schematically, the different steps of a variant of amethod for enriching a learning base, according to the invention;

FIG. 4 represents, schematically, the different steps of a method forglobal analysis of an image, according to the invention;

FIG. 5 represents, schematically, the different steps of a step C) of amethod for enriching a learning base, according to the invention;

FIGS. 6 and 18 represent, schematically, the different steps of a methodfor modeling the dental arch of a patient, according to the invention;

FIG. 7 represents, schematically, the different steps of a method forassessing a dental situation of a patient, according to the invention;

FIG. 8 represents, schematically, the different steps of a method foracquiring an image of a dental arch of a patient, according to theinvention;

FIG. 9 represents, schematically, the different steps of a method forassessing the shape of an orthodontic aligner of a patient, according tothe invention;

FIG. 10 represents an example of a reference image of an initialreference model;

FIG. 11 (11A-11D) illustrates a way of processing to determine the toothmodels in an initial reference model, as described in WO 2016 066651,with FIG. 11A representing a reference image, FIG. 11B representing acorresponding reference map, FIG. 11C representing a map relating to thecontour of the teeth, and figure LID representing a segmented referencemodel;

FIG. 12 (12A-12D) illustrates the acquisition of an image by means of aretractor, an operation of cropping of this image, and the processing ofan updated image making it possible to determine the contour of theteeth, as described in WO 2016 066651, with FIG. 12A illustrating theuse of a dental retractor, FIGS. 12B and 12C representing an updatedimage, and FIG. 12D representing an updated map relating to the contourof the teeth obtained from the updated image of FIG. 12C;

FIG. 13 schematically illustrates the relative position of registrationmarks 12 of a retractor 10 on updated images 141 and 142, according tothe directions of observation represented by broken lines;

FIGS. 14 and 15 represent an orthodontic aligner, in perspective andplan views, respectively;

FIG. 16 illustrates the step e) described in WO 2016 066651;

FIG. 17 (17A-17C) illustrates an enrichment method according to theinvention, with FIG. 17A showing an example of updated image, FIG. 17Bshowing the reference image exhibiting a maximal match with the updatedimage, and FIG. 17C illustrating the transfer of the tooth numbers tothe updated tooth zones; and

FIG. 18 is described above with FIG. 6.

DETAILED DESCRIPTION

A detailed analysis method according to the invention requires thecreation of a learning base. This creation preferably implements amethod comprising the steps A) to F), or, in one embodiment, in place ofthe steps A) to C), preferably the steps A′) to C′).

First Main Embodiment of the Enrichment Method

The step A) is intended for the production of an updated reference modelmodeling an arch of the patient. It preferably comprises one or more ofthe features of the step a) of WO 2016 066651, incorporated byreference, to produce an initial reference model.

The updated reference model is preferably created with a 3D scanner.Such a so-called “3D” model may be observed from any angle. Anobservation of the model, from a determined angle and distance, iscalled a “view” or “reference image”.

FIG. 11A is an example of reference image.

The updated reference model may be prepared from measurements performedon the teeth of the patient or on a molding of his or her teeth, forexample a plaster molding.

For each tooth, from the updated reference model, a model of said tooth,or “tooth model”, is defined (FIG. 11D). This operation, known per se,is called “segmentation” of the updated reference model.

In the updated reference model, a tooth model is preferably delimited bya gingival margin which may be broken down into an interior gingivalmargin (of the side of the interior of the mouth relative to the tooth),an exterior gingival margin (oriented toward the outside of the mouthrelative to the tooth) and two lateral gingival margins.

One or more tooth attributes are associated with the tooth models as afunction of the teeth that they model.

The tooth attribute is preferably chosen from a tooth number, a toothtype, a tooth shape parameter, for example a tooth width, in particulara mesio-palatine width, a thickness, a crown height, an index of mesialand distal deflection of the incisive margin, or a level of abrasion, atooth appearance parameter, in particular a translucency index or acolor parameter, a parameter relating to the condition of the tooth, forexample “abraded”, “broken”, “decayed” or “fitted” (that is to say incontact with a dental, preferably orthodontic, appliance), an age forthe patient, or a combination of these attributes. A tooth attribute ispreferably an attribute which only relates to the tooth modeled by thetooth model.

A tooth attribute value may be assigned to each tooth attribute of aparticular tooth model.

For example, the “tooth type” tooth attribute will have the value“incisor”, “canine” or “molar” depending on whether the tooth model isthat of an incisor, of a canine or of a molar, respectively.

The assignment of the tooth attribute values to the tooth models may bemanual or, at least partly, automatic. For example, if the value of atooth attribute is identical whatever the tooth model, such as for the“age of the patient” tooth attribute, it may be sufficient to assign avalue to one tooth model to determine the value of this attribute forthe other tooth models.

Similarly, the tooth numbers are conventionally assigned according to astandard rule. It is therefore sufficient to know this rule and thenumber of a tooth modeled by a tooth model to calculate the numbers ofthe other tooth models.

In a preferred embodiment, the shape of a particular tooth model isanalyzed so as to define its tooth attribute value, for example itsnumber. This shape recognition is preferably performed by means of adeep learning device, preferably a neural network. Preferably, a libraryof historical tooth models is created, each historical tooth modelhaving a value for the tooth attribute, as described hereinbelow (stepa)), the deep learning device is trained with views of the historicaltooth models of this library, then one or more views of the particulartooth model are analyzed with the trained deep learning device, so as todetermine the tooth attribute value of said particular tooth model.

The allocation of the tooth attribute values may then be performedtotally without human intervention.

The step B) is intended for the acquisition of one or preferably severalupdated images. The step B) preferably comprises one or more of thefeatures of the step b) of WO 2016 066651.

The acquisition of the updated images is performed by means of an imageacquisition apparatus, preferably chosen from a cellphone, a so-called“connected” camera, a so-called “smart” watch, a tablet or a personalcomputer, fixed or portable, comprising an image acquisition system suchas a webcam or a camera. Preferably, the image acquisition apparatus isa cellphone.

Also preferably, the image acquisition apparatus is separated from thedental arch by more than 5 cm, more than 8 cm, even more than 10 cm,which avoids the condensation of steam on the optic of the imageacquisition apparatus and facilitates the focusing. Furthermore,preferably, the image acquisition apparatus, in particular thecellphone, has no specific optic for the acquisition of the updatedimages, which is notably possible because of the separation of thedental arch during the acquisition.

Preferably, an updated image is a photograph or an image extracted froma film. It is preferably in color, preferably in real colors.

Preferably, the acquisition of the updated image or images is performedby the patient, preferably without the use of a support which is bearingon the ground and is immobilizing the image acquisition apparatus, andin particular without tripod.

In one embodiment, the triggering of the acquisition is automatic, thatis to say without the action of an operator, as soon as the acquisitionconditions are approved by the image acquisition apparatus, inparticular when the image acquisition apparatus has determined that itobserves a dental arch and/or a retractor and that the conditions ofobservation are satisfactory (sharpness, brightness, even dimensions ofthe representation of the dental arch and/or of the retractor).

The time interval between the steps A) and B) is as short as possible inorder for the teeth not to be substantially displaced between theproduction of the updated model and the acquisition of the updatedimages. Reference images matching with the updated images may then beacquired by observing the updated reference model.

Preferably, a dental retractor 10 is used in the step B), as representedin FIG. 12A. The retractor conventionally comprises a support providedwith a rim extending around an opening and arranged in such a way thatthe lips of the patient may rest thereon leaving the teeth of thepatient apparent through said opening.

In the step C), the updated reference model is explored to find, foreach updated image, a reference image exhibiting a maximal match withthe updated image.

The step C) may comprise one or more of the features of the steps c), d)and e) of WO 2016 066651, in as much as they concern such anexploration.

For each updated image, a set of virtual acquisition conditionsapproximating the real acquisition conditions upon the acquisition ofsaid updated image is preferably determined, roughly. In other words,the position of the image acquisition apparatus is estimated in relationto the teeth at the moment when it took the updated image (position ofthe acquisition apparatus in space and orientation of this apparatus).This rough assessment advantageously makes it possible to limit thenumber of tests on virtual acquisition conditions in subsequentoperations, and therefore makes it possible to considerably speed upthese operations.

To perform this rough assessment, one or more heuristic rules arepreferably used. For example, preferably, virtual acquisition conditionslikely to be tested in subsequent operations, the conditions whichcorrespond to a position of the image acquisition apparatus behind theteeth or at a distance from the teeth greater than 1 m, are excluded.

In a preferred embodiment, as illustrated in FIG. 13, registration marksare used that are represented on the updated image, and in particularregistration marks 12 of the retractor, to determine a region of thespace that is substantially conical delimiting virtual acquisitionconditions likely to he tested in subsequent operations, or “test cone”.

More specifically, at least three registration marks 12 are preferablypositioned non-aligned on the retractor 10, and their relative positionson the retractor are accurately measured.

The registration marks are then marked on the updated image, asdescribed previously. Simple trigonometric calculations make it possibleto determine approximately the direction from which the updated imagewas taken.

Next, for each updated image, a reference image is sought that exhibitsa maximum match with the updated image. This search is preferablyperformed by means of a metaheuristic method, preferably evolutionist,preferably by simulated annealing.

Preferably, at any instant before the step C4), the updated image isanalyzed so as to produce an updated map representing, at leastpartially, a discriminating information item. The updated map thereforerepresents the discriminating information item in the reference frame ofthe updated image.

The discriminating information item is preferably chosen from the groupconsisting of a contour information item, a color information item, adensity information item, a distance information item, a brightnessinformation item, a saturation information item, information onreflections and combinations of these information items.

A person skilled in the art knows how to process an updated image toreveal the discriminating information item.

For example, FIG. 12D is an updated map relating to the contour of theteeth obtained from the updated image of FIG. 12B.

Said search comprises the following steps:

-   -   C1) determination of virtual acquisition conditions “to be        tested”;    -   C2) production of a reference image of the updated reference        model said virtual acquisition conditions to be tested;    -   C3) processing of the reference image to produce at least one        reference map representing, at least partially, the        discriminating information item;    -   C4) comparison of the updated and reference maps so as to        determine a value for an assessment function, said value for the        assessment function depending on the differences between said        updated and reference maps and corresponding to a decision to        continue or stop the search for virtual acquisition conditions        approximating said real acquisition conditions with greater        accuracy than said virtual acquisition conditions to be tested        determined in the last occurrence of the step C1);    -   C5) if said value for the assessment function corresponds to a        decision to continue said search, modification of the virtual        acquisition conditions to be tested, then return to the step        C2).

The step C1) begins with a determination of the virtual acquisitionconditions to be tested, that is to say a virtual position andorientation likely to correspond to the real position and orientation ofthe acquisition apparatus upon the capture of the updated image, butalso, preferably, a virtual calibration likely to correspond to the realcalibration of the acquisition apparatus upon the capture of the updatedimage.

The first virtual acquisition conditions to he tested are preferablyvirtual acquisition conditions assessed roughly, as describedpreviously.

In the step C2), the image acquisition apparatus is then virtuallyconfigured in the virtual acquisition conditions to be tested in orderto acquire a reference image of the updated reference model in thesevirtual acquisition conditions to be tested. The reference imagetherefore corresponds to the image that the image acquisition apparatuswould have taken if it had been placed, relative to the updatedreference model, and optionally calibrated, in the virtual acquisitionconditions to be tested.

If the updated image was taken substantially at the same moment as theupdated reference model was created by a scan of the teeth of thepatient, the position of the teeth on the updated image is substantiallyidentical to that in the updated reference model. If the virtualacquisition conditions to be tested are exactly the real acquisitionconditions, the reference image may therefore he exactly superimposed onthe updated image. The differences :25 between the updated image and thereference image result from errors in the assessment of the virtualacquisition conditions to be tested, if they do not correspond exactlyto the real acquisition conditions.

In the step C3), the reference image is processed as the updated imageso as to produce, from the reference image, a reference map representingthe discriminating information item (FIGS. 11A and 11B). A personskilled in the art knows how to process a reference image to reveal thediscriminating information item.

In the step C4), the updated and reference maps, both relating to thesame discriminating information item, are compared and the difference or“distance” between these two maps is assessed by means of a score. Forexample, if the discriminating information item is the contour of theteeth, the mean distance between the points of the contour of the teethwhich appears on the reference image and the points of the correspondingcontour which appears on the updated image may be compared, the scorebeing all the higher when this distance is small.

The score may for example be a correlation coefficient.

Preferably, the virtual acquisition conditions comprise the calibrationparameters of the acquisition apparatus. The score is that much higherwhen the values of the calibration parameters tested are close to thecalibration parameter values of the acquisition apparatus used upon theacquisition of the updated image. For example, if the diaphragm aperturetested is far from that of the acquisition apparatus used upon theacquisition of the updated image, the reference image shows fuzzyregions and sharp regions which do not correspond to the fuzzy regionsand to the sharp regions of the updated image. If the discriminatinginformation item is the contour of the teeth, the updated and referencemaps will not therefore represent the same contours and the score willbe low.

The score is then assessed by means of an assessment function. Theassessment function makes it possible to decide whether the cycling onthe steps C1) to C5) must be continued or stopped. The assessmentfunction may for example be equal to 0 if the cycling must be stopped orbe equal to 1 if the cycling must continue.

The value of the assessment function may depend on the score achieved.For example, a decision may be made to continue the cycling if the scoredoes not exceed a threshold. For example, if an exact match between theupdated and reference images leads to a score of 100%, the threshold maybe, for example, 95%. Obviously, the higher the threshold, the betterwill be the accuracy of the assessment of the virtual acquisitionconditions if the score manages to exceed this threshold.

The value of the assessment function may also depend on scores obtainedwith virtual acquisition conditions tested previously.

The value of the assessment function may also depend on randomparameters and/or on the number of cycles already performed.

In particular, it is possible, despite the repetition of the cycles, notto manage to find virtual acquisition conditions which are sufficientlyclose to the real acquisition conditions for the score to reach saidthreshold. The assessment function may then lead to the decision to exitthe cycle even through the best score obtained has not reached saidthreshold. This decision may result, for example, from a number ofcycles greater than a predetermined maximum number.

A random parameter in the assessment function may also authorize thecontinuation of tests of new virtual acquisition conditions, even thoughthe score appears satisfactory.

The assessment functions conventionally used in the metaheuristicoptimization, preferably evolutionist, methods, particularly in thesimulated annealing methods, may be used for the assessment function.

In the step C5), if the value of the assessment function indicates adecision to continue the cycling, the virtual acquisition conditions tobe tested are modified and the cycling recommences on the steps C1) toC5), consisting in producing a reference image and a reference map, incomparing the reference map with the updated map to determine a score,then in taking a decision as a function of this score.

The modification of the virtual acquisition conditions to be testedcorresponds to a virtual displacement in space and/or to a modificationof the orientation and/or, preferably, to a modification of thecalibration of the acquisition apparatus. This modification may berandom, preferably such that the new virtual acquisition conditions tobe tested still belong to the set determined upon the rough assessment.The modification is preferably guided by heuristic rules, for example byfavoring the modifications which, according to an analysis of thepreceding scores obtained, appear most favorable for increasing thescore.

The cycling is continued until the value of the assessment functionindicates a decision to cease this cycling and to continue to the stepD), for example if the score reaches or exceeds said threshold.

The optimization of the virtual acquisition conditions is preferablyperformed by using a metaheuristic, preferably evolutionist, method,preferably a simulated annealing algorithm. Such an algorithm is wellknown for nonlinear optimization.

If the cycling has been exited without a satisfactory score having beenable to be obtained, for example without the score having been able toreach said threshold, the method may be stopped (failure situation) or anew step C) may be launched, with a new discriminating information itemand/or with a new updated image. The method may also be continued withthe virtual acquisition conditions corresponding to the best scorereached. A warning may be emitted in order to inform the user of theerror on the result.

If the cycling has been exited when a satisfactory score has been ableto be obtained, for example because the score has reached, even exceededsaid threshold, the virtual acquisition conditions correspondsubstantially to the real acquisition conditions of the updated image.

Preferably, the virtual acquisition conditions comprise the calibrationparameters of the acquisition apparatus. The method thus makes itpossible to assess the values of these parameters without it beingnecessary to know the nature of the acquisition apparatus or itssetting. The acquisition of the updated images may therefore beperformed without any particular precautions, for example by the patienthim or herself by means of his or her cellphone.

In addition, the search for the real calibration is performed bycomparing an updated image with views of a reference model in virtualacquisition conditions to be tested. Advantageously, it does not requirethe updated image to show a standard calibration gauge, that is to say agauge whose characteristics are accurately known making it possible todetermine the calibration of the acquisition apparatus.

The step C) therefore culminates in the determination of virtualacquisition conditions exhibiting a maximal match with the realacquisition conditions. The reference image therefore exhibits a maximalmatch with the updated image, that is to say that these two images maysubstantially be superimposed.

In one embodiment, said search of the virtual acquisition conditions inthe step C) is performed by using a metaheuristic, preferablyevolutionist, method, preferably a simulated annealing algorithm.

In the step D), the reference tooth zones are identified on thereference image and they are transferred to the updated image to definecorresponding updated tooth zones.

In particular, the reference image is a view of the updated referencemodel segmented into tooth models. The limits of the representation ofeach tooth model on the reference image, or “reference tooth zone”, maytherefore he identified.

The superimposition of the updated and reference images then makes itpossible to transfer the limits of the reference tooth zones to theupdated image, and thus define the updated tooth zones. Since thereference image exhibits a maximal match with the updated image, theupdated tooth zones therefore substantially define the limits of thetooth models represented on the reference image.

In the step E), each updated tooth zone is assigned the tooth attributevalue or values of the tooth model which corresponds to it.

In particular, the reference image is a view of the updated referencemodel in which the tooth models have been assigned respective toothattribute values for at least one tooth attribute, for example a toothnumber. Each reference tooth zone may therefore inherit the toothattribute value of the tooth model that it represents. Each updatedtooth zone may then inherit the tooth attribute value of the referencetooth zone which allowed it to be defined.

At the end of the step E), an updated image and a description of theupdated image is therefore obtained that defines one or more updatedtooth zones and, for each of these zones, a tooth attribute value for atleast one tooth attribute, for example a tooth number.

The updated image enriched with its description is called “historicalimage”.

FIG. 17A shows an example of updated image (acquired in the step B))being analyzed in order to determine the contours of the teeth. FIG. 17Bshows the reference image exhibiting a maximal match with the updatedimage (resulting from the step C)). The numbers of the teeth aredisplayed on the corresponding teeth. FIG. 17C illustrates the transferof the tooth numbers to the updated tooth zones (step I)) and E)).

In the step F), the historical image is added to the learning base.

The steps A) to F) are preferably performed for more than 1000, morethan 5000, or more than 10000 different patients, or “historicalpatients”.

As now clearly emerges, the invention provides a method that isparticularly effective for creating a learning base.

The invention relates also to a method for training a deep learningdevice, preferably a neural network, said method comprising anenrichment of a learning base according to a method comprising steps A)to F) so as to acquire a plurality of historical images, then the use ofsaid learning base to train said deep learning device.

Second Main Embodiment of the Enrichment Method

The invention is not however limited to the embodiments described above.

In particular, the updated reference model is not necessarily the directresult of a scan of the arch of the patient. The updated reference modelmay in particular be a model obtained by deformation of an initialreference model which is itself resulting directly from such a scan.

The method then comprises, preferably, in place of the steps A) to C),the steps A′) to C′).

The step A′) is identical to the step A). In the step A′), the referencemodel generated is however intended to be modified. It is thereforequalified as “initial reference model”, and not as “updated referencemodel”, as according to the step A).

The initial reference model may in particular be generated at an initialinstant preceding an active orthodontic treatment, for example less than6 months, less than 3 months, or less than 1 month before the start ofthe treatment. The steps B′) to C′) may then be implemented to track thetrend of the treatment between the initial instant and the updatedinstant of the step B′).

The initial instant may alternatively be an instant at the end of activeorthodontic treatment, for example less than 6 months, less than 3months, or less than 1 month after the end of the treatment. The stepsB′) to C′) may then be implemented to monitor the appearance of anyrecurrence.

The step B′) is identical to the step B). In the step B′), the updatedimages are however also intended to guide the modification of theinitial reference model to define the updated reference model, in thestep C′).

The time interval between the steps A′) and B′) is not mited, since, asexplained hereinbelow, the initial reference model will be deformed toobtain an updated reference model with maximal match with the updatedimages. The time interval between the steps A′) and B′) may for examplebe greater than 1 week, than 2 weeks, than 1 month, than 2 months ormore than 6 months.

The step C′) is more complex than the step C) since the search for areference image exhibiting a maximal match with an updated image is notlimited to searching for the optimal virtual acquisition conditions. Italso includes a search for an updated reference model, that is to say areference model in which have substantially the same position as on theupdated image.

The step C′) preferably comprises one or more of the features of thesteps c), d) and e) of WO 2016 066651, and in particular of the step e)illustrated in FIG. 16.

The objective is to modify the initial reference model until an updatedreference model is obtained which exhibits a maximal match with theupdated image. Ideally, the updated reference model is therefore a modelof the arch from which the updated image would have been able to betaken if this model had been the arch itself.

A succession of reference models “to be tested” is therefore tested, thechoice of a reference model to be tested preferably being dependent onthe level of correspondence of the reference models “to be tested”previously tested with the updated image.

Preferably, the search comprises, for an updated image,

-   -   a first optimization operation making it possible to search, in        a reference model to be tested determined from the initial        reference model, for virtual acquisition conditions best        corresponding to the real acquisition conditions of the updated        image, and    -   a second optimization operation making it possible to search, by        testing a plurality of said reference models to be tested, for        the reference model best corresponding to the positioning of the        teeth of the patient upon the acquisition of the updated image.

Preferably, a first optimization operation is performed for each test ofa reference model to be tested during the second optimization operation.

Preferably, the first optimization operation and/or the secondoptimization operation, preferably the first optimization operation andthe second optimization operation, implement a metaheuristic method,preferably evolutionist, preferably a simulated annealing.

The step C′) therefore culminates in the determination

-   -   of an updated reference model exhibiting a maximal match with        the updated image, and    -   virtual acquisition conditions exhibiting a maximal match with        the real acquisition conditions. A method comprising the steps        A′) to C′) may advantageously be implemented in the context of        an active or passive orthodontic treatment, or, more generally,        to track any changes to the teeth.

In the different methods according to the invention, the enrichment ofthe learning base does not necessarily result from an enrichment methodaccording to the invention.

In one embodiment, the learning base is created by an operator. Thelatter thus analyzes thousands of analysis images. For the learning baseto be able to be used for the implementation of a detailed analysismethod, it determines the tooth zones, then assigns them tooth attributevalues. For the learning base to be able to be used for theimplementation of a global analysis method, it assigns image attributevalues to each image. It may thus form historical images.

Detailed Image Analysis Method

The method for detailed analysis of an “analysis image” of a dental archof a patient according to the invention comprises the steps 1) to 4).

Preferably, the analysis image, preferably a photograph or an imageextracted from a film, preferably in color, preferably in real color, isacquired with an image acquisition apparatus, preferably a cellphone,separated from the dental arch by more than 5 cm, more than 8 cm, evenmore than 10 cm, and which, preferably, has no other specific optic.

Preferably, the analysis image represents several teeth, preferably morethan 2, more than 3, more than 4 or more than 5 teeth of the patient.FIG. 12A or FIG. 12B could be examples of analysis images. Thearrangement of the teeth is realistic, i.e. it corresponds to thearrangement which is observed by the image acquisition apparatus when ithas acquired the analysis image.

Preferably, the acquisition of the analysis image is performed by thepatient, preferably without the use of a support which is bearing on theground and is immobilizing the image acquisition apparatus, and inparticular without tripod.

In one embodiment, the triggering of the acquisition of the analysisimage is automatic, that is to say without the action of an operator, assoon as the acquisition conditions are approved by the image acquisitionapparatus, in particular when the image acquisition apparatus hasdetermined that it observes a dental arch and/or a retractor and thatthe observation conditions are satisfactory (sharpness, brightness, evendimensions of the representation of the dental arch and/or of theretractor).

In the step 1), a learning base is created comprising more than 1000,preferably more than 5000, preferably more than 10 000, preferably morethan 30 000, preferably more than 50 000, preferably more than 100 000historical images. The greater the number of historical images, thebetter the analysis performed by the method.

Preferably, a learning base is used that is enriched according to anenrichment method according to the invention.

The learning base may however be constructed according to other methods,for example be created manually. To create a historical image of thelearning base, an operator, preferably an orthodontist, identifies oneor more “historical” tooth zones on an image, called then assigns, toeach identified historical tooth zone, a value for at least one toothattribute.

In step 2), a deep learning device, preferably a neural network, istrained with the learning base.

A “neural network” or “artificial neural network” is a set of algorithmswell known to a person skilled in the art.

The neural network may in particular be chosen from:

-   -   the networks specializing in the classification of images,        called “CNN” (“convolutional neural network”), for example        -   AlexNet (2012)        -   LF Net (2013)        -   VGG Net (2014)        -   GoogleNet (2015)        -   Microsoft ResNet (2015)        -   Caffe: BAIR Reference CaffeNet, BAIR AlexNet        -   Torch:VGG_CNN_S, VGG_CNN_M, VGG_M_2048, VGG_CNN_M_10            24,VGG_CNN_M_128,VGG_CNN_F,VGG ILSVRC-2014 16-layer,VGG            ILSVRC-2014 19-layer,Network-in-Network magenet & CIFAR-10)        -   Google: Inception (V3, V4).    -   The networks specializing in the location and detection of        objects in an image, the object detection network, for example:        -   R-CNN (2013)        -   SSD (Single Shot MultiBox Detector: Object Detection            network), Faster R--CNN (Faster Region-based Convolutional            Network method: Object Detection network)        -   Faster R-CNN (2015)        -   SSD (2015).

The above list is not limiting

In the step 2), the deep learning device is preferably trained by alearning process called “deep learning”. By presenting, as input for thedeep learning device, historical images (images+descriptions), the deeplearning device progressively learns to recognize patterns on an image,and to associate them with tooth zones and with tooth attribute values,for example tooth numbers.

In the step 3), the image that is to be analyzed, or “analysis image”,is submitted to the deep learning device.

Through its training in the step 2), the deep learning device is capableof analyzing the analysis image and of recognizing said patternstherein. It may in particular determine a probability relating to:

-   -   the presence, at a location in said analysis image, of a zone        representing, at least partially, a tooth, or “analysis tooth        zone”,    -   the attribute value of the tooth represented on said analysis        tooth zone.

For example, it is capable of determining that there is a 95% chancethat a form of the analysis image represents an incisor.

Preferably, the deep learning device analyses all the analysis image anddetermines probabilities for all the analysis tooth zones that it hasidentified.

In the step 4), the results of the preceding step are analyzed todetermine the teeth represented on the analysis image.

When the learning base comprises more than 10 000 historical images, thestep 3) gives particularly satisfactory results. In particular, such alearning base makes it possible to establish a probability thresholdsuch that, if a probability associated with an analysis tooth zone andwith a tooth attribute value for this analysis tooth zone exceeds saidthreshold, the analysis tooth zone effectively represents a tooth havingsaid tooth attribute value.

The step 4) thus leads to the definition of an analysis image enrichedwith a description defining the analysis tooth zones and, for eachanalysis tooth zone, the values of the attributes of the toothrepresented by the analysis tooth zone.

Global Image Analysis Method

The method for global analysis of an updated image of a dental arch of apatient according to the invention comprises the steps 1′) to 3′).

The method is similar to the detailed analysis method described above,apart from the fact that, according to the global analysis, it is notnecessary to analyze die individual situation of each tooth. Theanalysis is global to all the image. In other words, the deep learningdevice determines the value of an “image” attribute without having topreviously determine tooth attribute values.

For example, the deep learning device may conclude that, “globally”, thedental situation is “satisfactory” or “unsatisfactory”, withoutdetermining the tooth potentially at the origin of the dissatisfaction.

The step 1′) is similar to the step 1). The historical images howeverinclude a description specifying an image attribute value for eachimage.

The step 2′) is similar to the step 2).

In the step 3′), the analysis image is submitted to the deep learningdevice.

Through its training in the step 2′), the deep learning device iscapable of analyzing the analysis image and of recognizing said patternstherein. Based on these patterns, it may in particular determine aprobability relating to the value of the image attribute considered.

Application to the Modeling of a Dental Arch

A detailed analysis method according to the invention is in particularuseful for modeling a dental arch, notably for establishing a remotediagnostic.

It is desirable for each to regularly check his or her definition,particularly in order to check that the position of his or her teeth isnot changing for the worse. In an orthodontic treatment, this change forthe worse may in particular lead to the treatment being modified. Afteran orthodontic treatment, a change for the worse, called “recurrence”,may lead to a repeat of a treatment. Finally, more generally andindependently of any treatment, each may want to track any displacementsof his or her teeth.

Conventionally, the checks are performed by an orthodontist who hasappropriate apparatus. These checks are therefore costly. Furthermore,the appointments are restrictive. Finally, some people are apprehensiveof a visit to an orthodontist and will fail to make an appointment for asimple check or to assess the feasibility of an orthodontic treatment.

US 2009/0291417 describes a method making it possible to create, thenmodify, three-dimensional models, particularly for the manufacture oforthodontic appliances.

WO 2016 066651 describes a method for checking the positioning and/orthe shape and/or the appearance of teeth of a patient. This methodcomprises a step of creation of an initial reference model of the teeth,at an initial instant, preferably with a 3D scanner, then, at asubsequent instant, or “updated instant”, for example six months afterthe initial instant, the creation of an updated reference model, bydeformation of the initial reference model. This deformation isperformed in such a way that the updated reference model allowsobservation substantially identical to images of the teeth acquired atthe updated instant, in particular to photos or images from a videotaken by the patient him or herself, with no particular precautions,called “updated images”.

The updated images are therefore used to modify the initial, veryaccurate, reference model. The updated reference model which resultsfrom the deformation of the initial reference model, guided by theanalysis of the updated images, is therefore also very accurate.

The method described in WO 2016/66651 does however require anappointment with the orthodontist in order to create the initialreference model. This appointment constitutes a brake to prevention. Ineffect, a patient will not necessarily consult an orthodontist if he orshe does not perceive the need therefor. In other words, the method isoften implemented only if a malocclusion is found and has to becorrected.

There is therefore a need for a method that addresses this problem, byfacilitating prevention.

One aim of the invention is to address this need.

To this end, the invention proposes a method for modeling a dental archof a patient, said method comprising the following steps:

-   -   a) creation of a historical library comprising more than 1000        tooth models, called “historical tooth models”, and assignment        to each historical tooth model of a value for at least one tooth        attribute, or “tooth attribute value”;    -   b) analysis of at least one “analysis image” of the dental arch        according to a detailed analysis method according to the        invention, so as to determine at least one analysis tooth zone        and at least one tooth attribute value associated with said        analysis tooth zone;    -   c) for each analysis tooth zone determined in the preceding        step, search, in the historical library, for a historical tooth        model exhibiting a maximum proximity with the analysis image or        with the analysis tooth zone, or “optimal tooth model”;    -   d) arrangement of all the optimal tooth models so as to create a        model which exhibits a maximal match with the updated image, or        “assembled model”;    -   e) optionally, replacement of at least one optimal tooth model        by another historical tooth model and repeat of the step d) so        as to maximize the match between the assembled model and die        analysis image;    -   f) optionally, repeat of the step b) with another analysis image        and, in the step and/or e), search for a maximal match with all        the analysis images used.

The invention thus makes it possible, from a simple analysis image, forexample a photograph taken by means of a cellphone, to reconstruct, witha good reliability, a dental arch in the form of an assembled model. Inparticular, the analysis image may be acquired as described here aboveat step 1).

Obviously, the analysis of a single analysis image is insufficient togenerate an assembled model which accurately corresponds to thearrangement of the teeth of the, patient. Such an accuracy is howevernot generally essential for performing a first diagnosis of the dentalsituation of the patient.

Moreover, the accuracy of the assembled model may be increased ifseveral analysis images are processed.

The steps b) to c) are preferably implemented for several analysisimages and, in the steps d) and e), the optimal tooth models and anassembled model are searched so as to obtain a maximal match in light ofall the analysis images (step f)).

The invention also relates to a method or assessing a dental situationof a patient, comprising the following steps:

-   -   i) creation of an assembled model according to a modeling method        according to the invention;    -   ii) transmission of the assembled model to a recipient,        preferably an orthodontist and/or a computer;    -   iii) analysis of the orthodontic situation of the patient, by        the recipient, from the assembled model;    -   iv) preferably, informing of the patient of the orthodontic        situation, preferably, via his or her cellphone.

The patient may therefore very easily ask an orthodontist to check hisor her dental situation, without even having to travel, by simplytransmitting one or preferably several photos of his or her teeth.

A modeling method is now described in detail.

In the step a), a historical library 20 (FIG. 18) is created comprisingmore than 1000, preferably more than 5000, preferably more than 10 000historical tooth models 22. The greater the number of historical toothmodels, the more accurate the assembled model.

A historical tooth model can in particular be obtained from a model of adental arch of a “historical” patient obtained with a scanner. This archmodel may be segmented in order to isolate the representations of theteeth, as in FIG. 11D. Each of these representations, exhibiting aspecific gray shade in FIG. 11D, can constitute a historical toothmodel.

Preferably, the library is enriched with the tooth models resulting fromthe implementation of the method described in WO 2016 066651 or of thestep A) or A′) described above.

One or several tooth attributes, in particular chosen from the listsupplied above, are associated with the tooth models. A tooth attributevalue is assigned to each tooth attribute of a particular tooth model,as described previously (see the description of the step A)). Forexample, a tooth model is that of an “incisor”, “greatly worn”, andwhose color parameters are, in the color system L,*a*b* according to thestandard NF ISO 7724, “a*=2”, “b*=1” and “L*=58”.

The historical library therefore contains historical tooth models andassociated attribute values which facilitate the search in the step c).In FIG. 18, only historical tooth models 22 representing molars havebeen represented in the historical library 20.

In the step b), an analysis image is acquired, as described above forthe step B), before analyzing it. In particular, the analysis image ispreferably a photo or an image from a film, preferably with a cellphone.

The analysis image can be acquired at any moment after the step a), forexample more than one week, more than one month or more than six monthsafter the step a).

The analysis image is analyzed according to a detailed analysis methodaccording to the invention. The optional features of this method arealso optional in the step b).

On completion of the step b), an analysis image is obtained that isenriched with a description providing, for each analysis tooth zone, atooth attribute value for at least one tooth attribute, for example atooth number.

In the step c), a search is carried out in the historical library, foreach analysis tooth zone determined in the preceding step, for ahistorical tooth model exhibiting a maximal proximity with the analysistooth zone. This tooth model is qualified as “optimal tooth model”.

The “proximity” is a measurement of one or more differences between thehistorical tooth model and the analysis tooth zone. These differencesmay include a difference in shape, but also other differences like adifference in translucency or in color. The maximal proximity may besearched for by successively minimizing several differences, or byminimizing a combination of these differences, for example a weightedsum of these differences.

The “proximity” is therefore a broader concept than “match”, the matchmeasuring only a shape-related proximity.

The assessment of the proximity of a historical tooth model with ananalysis tooth zone preferably comprises a comparison of at least onevalue of a tooth attribute of the analysis tooth zone with the value ofthis attribute for the historical tooth model. Such an assessment isadvantageously very rapid.

For example, if the description of the analysis tooth zone provides avalue for the type or the number of the tooth, the thickness of thetooth represented and/or the height of its crown and/or its mesiopalatine width and/or the mesial and distal deflection index of itsincisal edge, this value may be compared to the value of thecorresponding at rib to of each of the historical tooth models.

Preferably, a historical tooth model is sought that has, for at leastone tooth attribute, the same value as said analysis tooth zone. Thetooth attribute may in particular relate to the tooth type or to thetooth number. In other words, the historical tooth models are filteredto examine in more detail only those which relate to the same type oftooth as the tooth represented on the analysis tooth zone.

Alternatively or, preferably, in addition to this comparison of theattribute values, the shape of the tooth represented on the analysistooth zone may be compared to the shape of a historical tooth model tobe assessed, preferably by means of a metaheuristic method, preferablyevolutionist, preferably by simulated annealing.

To this end, the historical tooth model to be assessed is observed fromdifferent angles. Each view thus obtained is compared with the analysisimage, preferably with the analysis tooth zone, so as to establish a“distance” between this view and said analysis image or, preferably,said analysis tooth zone. The distance thus measures the differencebetween the view and the analysis tooth zone.

The distance may be determined after a processing of the view and of theanalysis image or, preferably, of the analysis tooth zone, so as toreveal, on corresponding maps, one and the same discriminatinginformation item, for example a contour information item, as describedabove in the step C3) or in WO 2016 066651.

For each historical tooth model tested, a view is thus determined whichprovides a minimal distance with the analysis image or with the analysistooth zone. Each historical tooth model examined is thus associated witha particular minimal distance, which measures its proximity of shapewith the analysis tooth zone.

The optimal historical tooth model is the one which, in light of thecomparison or comparisons performed, is considered to he closest to theanalysis tooth zone.

The minimal distances obtained for the different tooth models tested arethen compared and, to define the optimal tooth model, the one whichexhibits the smallest minimal distance is retained. The optimal toothmodel therefore exhibits a maximal match with the analysis image.

The search for the maximal match is preferably performed by means of ametaheuristic method, preferably evolutionist, preferably by simulatedannealing.

In a preferred embodiment, a first assessment of the historical toothmodels by comparison of the values of at least one tooth attribute, forexample the tooth number, with the corresponding values of the analysistooth zone, then a second assessment by comparison of shape areperformed in succession. The first, rapid assessment, advantageouslymakes it possible to filter the historical tooth models in order tosubmit only the historical tooth models retained by the first assessmentto the second, slower assessment.

For example, if an analysis tooth zone represents a tooth number 15, thefirst assessment makes it possible to retain only the tooth modelsmodeling teeth number 15. During the second assessment, a search isconducted, among all the historical tooth models modeling teeth number15, for the historical tooth model whose shape most approximates that ofthe tooth represented.

More preferably, several first assessments are performed beforeperforming the second assessment. For example, the first assessmentsmake it possible to filter the historical tooth models so as to retainonly the tooth models modeling teeth number 15 and which exhibit a crownheight of between 8 and 8.5 mm.

On completion of the step c), an optimal tooth model has thus beenassociated with each of the analysis tooth zones.

For example, in FIG. 18, the historical tooth model 22 ₁ can be observedso as to strongly resemble an analysis zone identified on the analysisimage. It is considered as optimal for this analysis zone.

In the step d), an assembled model is created by arranging the optimaltooth models.

According to one embodiment, at the start of the step d), a first rougharrangement is created, that is to say that a rough model is produced byassembling optimal tooth models.

To establish the first rough arrangement, the optimal tooth models canbe oriented so that their optimal directions of observation are allparallel, the optimal direction of observation of a tooth model beingthe direction in which said tooth model exhibits a maximum match withthe analysis image.

The first rough arrangement may be also established by considering thetooth attribute values of the optimal tooth models. For example, if thetooth numbers of optimal tooth models are those of the canines and ofthe incisors, these tooth models may be arranged according to an arc 24(FIG. 18) corresponding conventionally to the region of the arch whichbears these types of teeth.

The shape of this arc may be refined as a function of other toothattribute values.

The order of the optimal tooth models is that of the correspondinganalysis tooth zones.

Moreover, the minimal distance associated with an optimal tooth modelresults from an observation of the tooth model according to an “optimal”observation direction. In other words, it is probably substantially fromthis direction that the tooth that this model models is also observed inthe analysis image. All the optimal tooth models are thus preferablyoriented in such a way that their respective optimal observationdirections are all parallel.

It is thus possible to define a first arrangement of the optimal toothmodels.

Preferably, the first arrangement of the optimal tooth models is thenmodified iteratively, so as to exhibit a maximal match with the analysisimage.

To assess an arrangement, it is observed according to different angles.Each view thus obtained is compared with the analysis image so as toestablish a “distance” between this view and said analysis image. Thedistance thus measures the difference between the view and the analysisimage.

The distance may be determined after a processing of the view and of theanalysis image so as to reveal, on one of the corresponding maps, adiscriminating information item, for example a contour information item,as described above in the step C3) or in WO 2016 066651.

For each arrangement examined, a view is thus determined that provides aminimal distance with the analysis image. Each arrangement examined isthus associated with a minimal distance.

The minimal distances obtained for the different arrangements tested arethen compared and, to define the optimal arrangement, the one whichexhibits the smallest minimal distance is retained. The optimalarrangement thus exhibits a maximal match with the analysis image.

The search for the maximal match is performed preferably by means of ametaheuristic method, preferably evolutionist, preferably by simulatedannealing.

On completion of the step d), an optimal arrangement of the optimaltooth models, that is to say the assembled model 26, is obtained.

In the optional step e), one or more optimal tooth models are replacedby other tooth models, then there is a return to the step d) so as tomaximize the match between the assembled model and the analysis image.

It is in fact possible for an optimal tooth model, in the “optimal”arrangement, to no longer exhibit a maximal match with the analysisimage. In particular, the tooth model might have been observed in an“optimal” direction which provided a view exhibiting a minimal distancewith the analysis image (reason why it was considered as optimal), but,in the optimal arrangement, it is no longer oriented according to theoptimal direction.

A new search for an assembled model may therefore be performed bymodifying the tooth models, for example by replacing the optimal toothmodels with close tooth models.

The search for the tooth models to be tested is preferably performed bymeans of a metaheuristic method, preferably evolutionist, preferably bysimulated annealing.

In the preferred embodiment, the method therefore implements a doubleoptimization, on the tooth models and on the arrangement of the toothmodels, the assembled model being the arrangement of a set of toothmodels which provides the minimal distance with the analysis image, byconsidering all the possible tooth models and all the possiblearrangements.

In the optional and preferred step f), the method implements severalanalysis images of the arch of the patient, preferably more than 3, morethan 5, more than 10, more than 50, preferably more than 100 analysisimages. The assembled model is thus more complete. More preferably, themethod implements an optimization such that the assembled model obtainedis optimal in light of all of the analysis images. In other words, theassembled model is preferably the one which maximizes the match with allof the analysis images.

In the preferred embodiment, the method therefore implements a doubleor, preferably a triple optimization, on the tooth models on the onehand, on the arrangement of the tooth models and/or on a plurality ofanalysis images on the other hand, the assembled model being thearrangement of a set of tooth models which provides the mean minimaldistance, over all of the analysis images, by considering all thepossible tooth models and, preferably, all the possible arrangements.

According to one embodiment, in the step d) and/or e) and/or f), ametahemistic, preferably evolutionist, method, preferably by simulatedannealing, is used.

As now clearly appears, the invention thus makes it possible toconstruct an assembled dental arch model from simple analysis images,for example photographs taken by means of a cellphone. Obviously, theaccuracy of the assembled model does not reach that of a scan. In someapplications, for example to perform a first diagnosis of the dentalsituation of the patient, such an accuracy is not however essential.

The assembled model may therefore be used to analyze the orthodonticsituation of the patient, according to the steps ii) to iv).

In the step ii), the assembled model is sent to an orthodontist and/orto a computer provided with diagnostic software.

In one embodiment, the assembled model is sent accompanied by aquestionnaire filled out by the patient in order to improve the qualityof the analysis in the step iv).

In the step iii), the orthodontist and/or the computer examines theassembled model. Unlike an updated image, the assembled model allows anobservation from any angle. The analysis is advantageously moreaccurate.

In the step iv), the orthodontist and/or the computer informs thepatient, for example by sending him or her a message on his or hertelephone. This message may in particular inform the patient of anunfavorable situation and urge him or her to make an appointment withthe orthodontist.

The orthodontist may also compare the assembled model with assembledmodels received previously for the same patient. The analysis thereofadvantageously makes it possible to assess the trend of the situation.The message may thus inform the patient of an unfavorable change of hisor her situation, which improves the prevention.

The assembled model may also be compared with one or more modelsobtained by scanning of the teeth or from a molding of the teeth of thepatient, or with an updated reference model resulting from theimplementation of a method described in WO 2016 066651.

Application to Embedded Monitoring

An image analysis according to the invention is also useful for guidingthe acquisition of an image of a dental arch, in particular forestablishing a remote diagnosis.

In particular, WO 2016/066651 describes a method in which an initialreference model is deformed so as to obtain an updated reference modelallowing the acquisition of reference images exhibiting a maximal matchwith the “updated” images of the arch acquired at the updated instant.

The reference images are therefore views of the updated reference model,observed in virtual acquisition conditions which are the closestpossible matches with the real acquisition conditions implemented toacquire the updated images of the arch of the patient.

The search for these virtual acquisition conditions is preferablyperformed by means of metaheuristic methods.

To speed up this search, WO 2016/066651 recommends performing a firstrough assessment of the real acquisition conditions. For example,conditions which would correspond to a position of the acquisitionapparatus at a distance from the teeth greater than 1 meter are excludedfrom the search.

There is however an ongoing need to speed up the execution of the methoddescribed in WO 2016/066651, and in particular, to search, more rapidly,for the virtual acquisition conditions exhibiting a maximal match withthe real acquisition conditions implemented to acquire an updated imageof the arch of the patient.

One aim of the invention is to at least partially address this problem.

The invention proposes a method for acquiring an image of a dental archof a patient, said method comprising the following steps:

-   -   a′) activation of an image acquisition apparatus so as to        acquire an image, called “analysis image”, of said arch;    -   b′) analysis of the analysis image by means of a deep learning        device, preferably a neural network, trained by means of a        learning base,    -   preferably according to a detailed analysis method according to        the invention, so as to identify at least one analysis tooth        zone representing a tooth on said analysis image, and to        determine at least one tooth attribute value for said analysis        tooth zone, or according to a global analysis method according        to the invention;    -   c′) determination, for the analysis image, of a value for an        image attribute, said value being a function of said tooth        attribute value if a detailed analysis method according to the        invention has been implemented in the preceding step;    -   d′) optionally, comparison of said image attribute value with an        instruction;    -   e′) sending of an information message as a function of said        comparison.

In one embodiment, in the step b′), all said analysis tooth zones areidentified, and at least one tooth attribute value is determined foreach analysis tooth zone, and, in the step c′), the value for the imageattribute is determined as a function of said tooth attribute values.

In one embodiment, the step h′) comprises the following steps:

-   -   1) preferably before the step a′), creation of a learning base        comprising more than 1 000 dental arch images, or “historical        images”, each historical image comprising one or more zones each        representing a tooth, or “historical tooth zones”, to each of        which, for said tooth attribute, a tooth attribute value is        assigned;    -   2) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3) submission of the analysis image to the deep learning device        so that it determines at least one probability relating to:        -   the presence, at a location in said analysis image, of a            zone representing, at least partially, a tooth, or “analysis            tooth zone”,        -   the attribute value of the tooth represented on said            analysis tooth zone,    -   4) determination, as a function of said probability, of the        presence of a tooth at a position represented by said analysis        tooth zone, and of the attribute value of said tooth.

In one embodiment, the step b′) comprises the following steps:

-   -   1′) creation of a learning base comprising more than 1 000        dental arch images, or “historical images”, each historical        image comprising an attribute value for at least one image        attribute, or “image attribute value”;    -   2′) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3′) submission of the analysis image to the deep learning device        so that it determines, for said analysis image, at least one        probability relating to said image attribute value.

In one embodiment, to create a historical image of the learning base, anoperator, preferably an orthodontist,

-   -   identifies one more “historical” tooth zones on an image, then        assigns each identified historical tooth zone a value for at        least one tooth attribute, and/or    -   assigns an image a value for at least one tooth attribute.

In one embodiment, the information message is sent by the acquisitionapparatus.

As will be seen in more detail hereinafter in the description, anacquisition method according to the invention therefore makes itpossible to check whether an analysis image respects an instruction and,if it does not respect the instruction, to guide the operator in orderfor him or her to acquire a new analysis image. The method thereforeallows an “embedded monitoring”, preferably in the image acquisitionapparatus.

In particular, to implement the method of WO 2016/066651, there may be adesire to acquire updated images from different acquisition directions,for example a front image, a right image and a left image. These updatedimages, acquired in succession, may be classified accordingly. Thesearch for the virtual acquisition conditions exhibiting a maximal matchwith the real acquisition conditions is speeded up thereby.

In effect, the search may begin from virtual acquisition conditions inwhich the virtual acquisition apparatus is in front, to the left or tothe right of the updated reference model, depending on whether theupdated image considered is classified as a front, left or right image,respectively.

The operator, generally the patient, may however make a mistake in theacquisition of the updated images. In particular, he or she may forgetto take an updated image, for example the front view, or invert twoupdated images. Typically, the operator may take an image on the rightwhereas a left image is expected from him or her.

This inversion of the updated images may considerably slow down theirprocessing. For example, if the updated image is assumed to be an imagetaken on the left but it was in error taken on the right, said searchfor the optimal virtual acquisition conditions, that is to say thoseexhibiting a maximal match with the real acquisition conditions, willbegin from a starting point offering a left view of the reference model,whereas the optimal virtual acquisition conditions correspond to a rightview. The search will therefore be considerably slowed down thereby.

By virtue of the invention, each updated image is an analysis imagewhich may be analyzed and checked, preferably in real time.

For example, the acquisition method makes it possible to determine thatthe updated image has been “taken on the right” and compare this imageattribute value with the instruction which had been given to theoperator to take the updated image on the left. Since the attributevalue of the updated image (image taken on the right) does notcorrespond to the instruction (acquire an updated image on the left),the acquisition apparatus may immediately alert the operator in orderfor him or her to modify the acquisition direction.

An acquisition method is now described in detail.

In the step a′), the operator activates the image acquisition apparatusso as to acquire an analysis image.

In one embodiment, the operator triggers the acquisition apparatus so asto store the analysis image, preferably takes a photo or video of his orher teeth, preferably by means of a cellphone equipped with a camera.

The step a′) may be performed like the acquisition of the updated imagesin the step B) described above.

In another embodiment, the analysis image is not stored. In particular,the analysis image may be the image which, in real time, appears on thescreen of the cellphone of the operator, generally the patient.

In a first embodiment, in the step b′), the analysis image is analyzedaccording to a detailed analysis method according to the invention. Thisanalysis preferably leads to the assignment of a tooth attribute valueto each identified analysis tooth zone, for example to assigning a toothnumber to each of the analysis tooth zones.

In the step c′), an attribute value of the analysis image is determinedas a function of the tooth attribute values. The attribute value of theanalysis image may relate to its general orientation and may for exampletake one of the following three values: “right photo”, “left photo” and“front photo”. The attribute value of the analysis image may also be thelist of the tooth numbers represented, for example, “16, 17 and 18”. Theattribute value of the analysis image may also be, for example, the“presence” or “absence” of a dental, preferably orthodontic, appliance,or the state of opening of the mouth (“mouth open”, “mouth closed”).

In another embodiment, a global analysis method according to theinvention is implemented in the step b′). Advantageously, such a methodmakes it possible to directly obtain a value for an image attribute,without having to determine values for a tooth attribute. It istherefore advantageously more rapid. The information resulting from aglobal analysis may however be less accurate than that resulting from adetailed analysis.

The steps a′) to c′) thus make it possible to characterize the analysisimage.

The characterization of the analysis image makes it possible to guidethe operator if the analysis image does not correspond to the expectedimage, for example because its quality is insufficient or because itdoes not represent the desired teeth.

In the step d′), the image attribute value of the analysis image iscompared with an instruction.

For example, if the instruction was to acquire a right image and theimage attribute value is “take on the left”, the comparison leads to theconclusion that the image acquired is “unsatisfactory”.

In the step e′), a message is sent to the operator, preferably by theacquisition apparatus.

Preferably, the information message relates to the quality of the imageacquired and/or to the position of the acquisition apparatus in relationto said arch and/or to the setting of the acquisition apparatus and/orto the opening of the mouth and/or to the wearing of a dental,preferably orthodontic, appliance.

For example, if the image acquired is “unsatisfactory”, the acquisitionapparatus may emit a light, for example red, and/or ring, and/orgenerate a voice message, and/or vibrate, and/or display a message onits screen.

For example, if the image has to be acquired while the patient wears hisor her dental appliance and this is not the case, the acquisitionapparatus may send the message “wear your appliance for this image”.

For example, if the image was acquired with the patient not sufficientlyopening the mouth or with the mouth closed, the acquisition apparatusmay send the message “open your mouth more for this image”.

In one embodiment, the steps b′) to c′) are implemented only if theoperator records the analysis image, that is to say he or she presses onthe trigger. The message then prompts the operator to acquire a newanalysis image. Optionally, the acquisition apparatus deletes theunsatisfactory analysis image.

In one embodiment, the steps b′) to c′) are implemented permanently whenthe acquisition apparatus is switched on and the analysis image is animage which appears on a screen of the acquisition apparatus. Theacquisition apparatus may thus, for example, emit a red light as long asthe analysis image is unsatisfactory, and emit a green light when theanalysis image is satisfactory. Advantageously, the acquisitionapparatus then stores only analysis images which are satisfactory.

As now clearly emerges, the invention therefore allows an embeddedmonitoring upon the acquisition of analysis images. When applied toupdated images of the method of WO 2016/066651, the steps a′) to e′)make it possible to check that these images do indeed conform to theneed, and therefore to considerably speed up the execution of thismethod.

The steps d′) and e′) are optional. In one embodiment, the analysisimage is only associated with its description, which specifies its imageattribute value. This description also makes it possible to considerablyspeed up the execution of the method of WO 2016/066651 since, when theanalysis image is used as updated image of this method, it makes itpossible to approximately determine the real acquisition conditions ofthis image, by eliminating the risk of a rough error, for examplebecause of a reversal between two images.

The steps d′) and e′) are however preferred. They make it possible forexample to avoid having the operator forget a left image, or take tworedundant right images.

Conventionally, at the start of an orthodontic treatment, theorthodontist determines the positioning of the teeth that he or shewants to obtain at a so-called “set-up” instant of the treatment. Theset-up may be defined by means of an imprint or from a three-dimensionalscan of the teeth of the patient. The orthodontist then hasmanufactured, or manufactures, accordingly, an orthodontic appliancesuited to this treatment.

The orthodontic appliance may be an orthodontic aligner. An alignerconventionally takes the form of a removable single-piece appliance,conventionally made of a transparent polymer material, which comprises achannel conformed for several teeth of an arch, generally all the teethof an arch, to be able to be housed therein.

The form of the channel is adapted to hold the aligner in position onthe teeth, while exerting an action of correction of the positioning ofcertain teeth (FIGS. 14 and 15).

At the start of the treatment, the shapes that the different alignershave to take at different moments of the treatment are conventionallydetermined, then the set of corresponding aligners is manufactured. Atpredetermined instants, the patient changes the aligner.

The treatment by means of aligners is advantageously less stressful forthe patient. In particular, the number of appointments with theorthodontist is limited. Furthermore, there is less pain than with ametal orthodontic brace attached to the teeth.

The market for orthodontic aligners is therefore on the increase.

At regular intervals, the patient goes to the orthodontist for a visualinspection, notably to check whether the displacement of the teethconforms to the expectations and whether the aligner worn by the patientis still suited to the treatment.

If the orthodontist diagnoses an unsuitability to the treatment, he orshe takes a new imprint of the teeth, or, in an equivalent manner, a newthree-dimensional scan of the teeth, then orders a new series ofaligners configured accordingly. It is considered that, on average, thenumber of aligners finally manufactured is approximately 45, instead of20 aligners conventionally provided at the start of the treatment.

The need to have to go to the orthodontist is a constraint for thepatient. The trust of the patient in his or her orthodontist may also beaffected. The inadequacy may be unsightly. Finally, the result thereofis an additional cost.

The number of inspection visits to the orthodontist must therefore belimited.

There is a need for solutions addressing these problems.

One aim of the invention is to at least partially address this need.

The invention provides a method for assessing the shape of anorthodontic aligner, said method comprising the following steps:

-   -   a″) acquisition of at least one image at least partially        representing the aligner in a service position in which it is        worn by a patient, called “analysis image”;    -   b″) analysis of the analysis image by means of a deep learning        device, preferably a neural network, trained by means of a        learning base, so as to determine a value for at least one tooth        attribute of an “analysis tooth zone” of the analysis image, the        tooth attribute relating to a separation between the tooth        represented by the analysis tooth zone, and the aligner        represented on the analysis image, and/or for an image attribute        of the analysis image, the image attribute relating to a        separation between at least one tooth represented on the        analysis image, and the aligner represented on said analysis        image;    -   c″) preferably, assessment of the suitability of the aligner as        a function of the value of said tooth or image attribute;    -   d″) preferably, sending of an information message as a function        of said assessment.

As will be seen in more detail hereinafter in the description, anassessment method according to the invention considerably simplifies theassessment of the good suitability of the aligner to the treatment,while making this assessment particularly reliable. In particular, themethod may be implemented from simple photographs or films, taken withno particular precautions, for example by the patient. The number ofappointments with the orthodontist may therefore be limited.

Preferably, in the step b″), all said analysis tooth zones areidentified, and the value of said tooth attribute is determined for eachanalysis tooth zone, and, in the step c″), the suitability of thealigner is determined as a function of said tooth attribute values.

Preferably, said tooth attribute is chosen from the group formed by amaximal separation along the free edge of the tooth, a mean separationalong the free edge of the tooth, and said image attribute is chosenfrom the group formed by a maximal separation along all of the teethrepresented, a mean separation along the free edges of all of the teethrepresented, an overall acceptability of the separation of the teethrepresented.

The tooth attribute relating to a separation may in particular be theexistence of a separation, this attribute being able to take the toothattribute values “yes” or “no”; or a value measuring the scale of theseparation, for example an observed maximal separation or an assessmentin relation to a scale.

In the step b″), a detailed analysis method according to the inventionis preferably implemented, a tooth attribute of each historical toothzone of each historical image of the learning base relating to aseparation between the tooth represented by the historical tooth zone,and an aligner worn by said tooth and represented on said historicalimage.

Preferably, the step b″) comprises the following steps:

-   -   b″1) preferably before the step a″), creation of a learning base        comprising more than 1000, preferably more than 5000, preferably        more than 10 000 images of dental arches, or “historical        images”, each historical image representing an aligner worn by a        “historical” patient and comprising one or more zones each        representing a tooth, or “historical tooth zones”, to each of        which is assigned, for at least one tooth attribute relating to        a separation between the tooth represented by the historical        tooth zone considered, and the aligner represented, a tooth        attribute value;    -   b″2) learning of a deep learning device, preferably a neural        network, by means of the learning base;    -   b″3) submission of the analysis image to the deep learning        device for the deep learning device to determine at least one        probability relating to        -   the presence, at a location of said analysis image, of an            analysis tooth zone, and        -   the tooth attribute value of the tooth represented on said            analysis tooth zone;    -   b″4) determination, as a function of said probability, of the        presence of a separation between the aligner and the tooth        represented by said analysis tooth zone, and/or of an amplitude        of said separation.

The steps b″1) to b″4) may comprise one or more of the features,possibly optional, of the steps 1) to 4) described above, respectively.

In one embodiment, in the step b″), a global analysis method accordingto the invention is implemented, an image attribute of each historicalimage of the learning base relating to a separation between at least onetooth represented on the historical image, and an aligner worn by saidtooth and represented on said historical image.

Preferably, the step b″) comprises the following steps:

-   -   b″1′) creation of a learning base comprising more than 1000        images of dental arches, or “historical images”, each historical        image comprising an attribute value for at least one image        attribute, or “image attribute value”, relating to a separation        between at least one tooth represented on the analysis image,        and the aligner represented on said analysis image;    -   b″2′) training of at least one deep learning device, preferably        a neural network, by means of the learning base;    -   b″3′) submission of the analysis image to the deep learning        device for it to determine, for said analysis image, at least        one probability relating to said image attribute value, and        determination, as a function of said probability, of the        presence of a separation between the aligner and the tooth or        teeth represented on the analysis image, and/or of an amplitude        of said separation.

The steps b″1′) to b″3′) may comprise one or more of the features,possibly optional, of the steps 1′) to 3′) described above,respectively.

The method is now described when a detailed analysis is implemented inthe step b″).

Prior to the step a″), the learning base must be enriched, preferablyaccording to an enrichment method according to the invention, in orderto contain historical images whose description specifies, for each ofthe historical tooth zones, a value for the tooth attribute relating tothe separation.

This information may be input manually. For example, an imagerepresenting one or more so-called “historical” tooth zones may bepresented to an operator, preferably an orthodontist, and he or she maybe asked to identify these historical tooth zones and indicate, for eachhistorical tooth zone, whether there is a separation or not and/orassess the amplitude of this separation.

A historical image may be a photo representing an aligner worn by ahistorical patient. Alternatively, a historical image may be the resultof a processing of an image representing a naked dental arch (that is tosay without aligner) and of an image representing the same arch bearingthe aligner. The image representing the naked arch may in particular bea view of a model of the arch deformed to obtain a maximal match withthe image representing the arch bearing the aligner. Such a processingmay in particular be useful for better revealing the outline of theteeth and of the aligner when the teeth are not very visible through thealigner.

In the step a″), the acquisition of the analysis image may be performedlike the acquisition of the updated images in the step B) describedabove.

Preferably, at least one reminder informing the patient of the need tocreate an analysis image is addressed to the patient. This reminder maybe in paper form or, preferably, in electronic form, for example in theform of an email, of an automatic alert from a dedicated mobile deviceor an SMS. Such a reminder may be sent by the orthodontic practice orlaboratory or by the dentist or by the dedicated mobile application ofthe patient, for example.

The step a″) is performed at the moment when the assessment of the shapeof an aligner is desired, for example more than 1 week after the startof the treatment with the aligner.

The analysis image is an image representing the aligner worn by theteeth of the patient.

In the step b″), the analysis image is analyzed according to a detailedanalysis method according to the invention.

The deep learning device has been trained by means of a learning basecontaining historical images whose description specifies, for at leastone, preferably each historical tooth zone, a value for a toothattribute relating to a separation between the tooth represented by thehistorical tooth zone and the aligner worn by said tooth and representedon said historical image.

The value for this tooth attribute therefore provides an informationitem relating to the shape of the aligner relative to the shape of theteeth of the patient.

The value for this tooth attribute can be a measurement of theseparation, for example a measurement of the maximum separation, or ofthe mean separation for the tooth represented by the historical toothzone.

The deep learning device is therefore capable of analyzing the analysisimage to determine, preferably for each of the “analysis tooth zones”,the existence, even the extent, of a separation of the aligner of thetooth represented on the analysis tooth zone.

In the step c″), an assessment is made, as a function of the results ofthe preceding step, as to the compatibility the aligner. For example, asearch is conducted to see if the separation of the aligner with atleast one tooth exceeds an acceptability threshold and, in this case, adecision is made as to the replacement of the aligner by a better suitedaligner.

The suitability of the aligner can be assessed in the context of anorthodontic treatment (separation compatible or not with the orthodontictreatment) but also in the context of a non-therapeutic, in particularesthetic, treatment. Aligners can indeed he used to displace teeth forpurely esthetic purposes, without this displacement modifying the stateof health of the patient. The suitability of the aligner can also beassessed in the context of a research program concerning theeffectiveness of the aligner, for example to assess a new material forthe aligner, on a human being or on another animal.

In the step d″), an information item relating to the assessment in thepreceding step is sent, in particular to the patient and/or theorthodontist.

The orthodontist can then use this information, possibly in combinationwith additional information, for example the age of the patient or theduration for which the aligner has been worn, to establish a diagnosisand, if necessary, decide on a suitable treatment.

In one embodiment, the method comprises, in the step b″), a globalanalysis according to the invention. The other steps are unchanged.

The analysis of the analysis image and of the historical images is thenperformed globally, without identification of the individual situationof each of the teeth represented and the image attribute relates to theimage as a whole.

For example, the image attribute relating to the separation may relateto the acceptability of a dental situation, the cause of one or moreseparations, or relate to the global scale of the separation orseparations of the teeth. For example, the value of the image attributemay be “globally acceptable” or “globally unacceptable”. The value ofthe image attribute can also, for example, be a measurement of theseparation, for example a measurement of the maximum separation, or ofthe mean separation between the teeth represented on the analysis imageand the aligner.

As now clearly emerges, a method according to the invention makes itpossible, from simple photos or a simple film, to determine whether thealigner is abnormally detached, even, if a detailed analysis wasperformed in the step b″), to determine the regions in which the aligneris separated from the teeth and assess the scale of this separation.

The invention also relates to a method for adapting an orthodontictreatment, a method in which a method for assessing the shape of anorthodontic aligner according to the invention is implemented, then, asa function of the result of said assessment, a new aligner ismanufactured and/or the patient is counseled, for example for him or herto improve the conditions of use of his or her orthodontic aligner, inparticular the positioning and/or the time bands when it should be wornand/or the maintenance of his or her orthodontic aligner, in order tooptimize the treatment.

The use of aligners is not limited to therapeutic treatments. Inparticular, an assessment method may be implemented to assess an alignerexclusively used for esthetic purposes.

The method may also be used to assess other dental, in particularorthodontic, parts or appliances.

Computer Program

The invention relates also:

-   -   to a computer program, and in particular a dedicated application        for cellphones, comprising program code instructions for the        execution of one or more steps of any method according to the        invention, when said program is run by a computer,    -   to a computer medium on which such a program is stored, for        example a memory or a CD-ROM.

Obviously, the invention is not limited to the embodiments describedabove and represented.

In particular, the patient is not necessarily a human being. A methodaccording to the invention may be used for another animal.

The patient could be alive or dead. He is preferably alive.

The methods of the invention may be used for an orthodontic treatment,but also out of any orthodontic treatment, and even out of anytherapeutic treatment.

1. A method for training a neural network, said method comprising anenrichment of a learning base so as to acquire a plurality of historicalimages, then the use of said learning base to train said neural network,said enrichment method comprising the following steps: A) at an“updated” instant, production of a digital three-dimensional model of adental arch of a patient, or “updated reference model”, and segmentationof the updated reference model so as to produce, for each tooth, adigital three-dimensional tooth, or “tooth model”, and, for at least onetooth attribute, assignment of a tooth attribute value to each toothmodel; B) acquisition of at least one image of said arch, or “updatedimage”, in real acquisition conditions; C) for each updated image,search for virtual acquisition conditions suitable for an acquisition ofan image of the updated reference model, called “reference image”,exhibiting a maximum match with the updated image in said virtualacquisition conditions, and acquisition of said reference image; D)identification, in the reference image, of at least one zonerepresenting a tooth model, or “reference tooth zone”, and, bycomparison of the updated image and the reference image, determination,in the updated image, of a zone representing said tooth model, or“updated tooth zone”; E) assignment, to said updated tooth zone, of thetooth attribute value or values of said tooth model; F) addition of theupdated image enriched with a description of said updated tooth zone andits tooth attribute value or values, or “historical image”, in thelearning base.
 2. The method as claimed in claim 1, in which said toothattribute is chosen from a tooth number, a tooth type, a tooth shapeparameter, a tooth appearance parameter, a parameter relating to thestate of the tooth, an age for the patient, or a combination of theseattributes.
 3. The method as claimed in claim 1, in which each updatedimage is processed to produce at least one updated map representing, atleast partially, a discriminating information item and, in the step C1,the search, for said updated image, comprises the following steps: C1)determination of virtual acquisition conditions to be tested; C2)production of a reference image of the updated reference model in saidvirtual acquisition conditions to be tested; C3) processing of thereference image to produce at least one reference map representing, atleast partially, the discriminating information item; C4) comparison ofthe updated and reference maps so as to determine a value for anassessment function, said value for the assessment function depending onthe differences between said updated and reference maps andcorresponding to a decision to continue or stop the search for virtualacquisition conditions approximating said real acquisition conditionswith greater accuracy than said virtual acquisition conditions to hetested determined on the last occurrence of the step C1); C5) if saidvalue for the assessment function corresponds to a decision to continuesaid search, modification of the virtual acquisition conditions to hetested, then return to the step C2).
 4. The method as claimed in claim1, in which the discriminating information item is a contour informationitem.
 5. The method as claimed in claim 4, in which said search for thevirtual acquisition conditions in the step C) is performed by using ametaheuristic method, preferably evolutionist, preferably a simulatedannealing algorithm.
 6. The method as claimed in claim 1, in which thesteps A) to C) are replaced by the following steps: A′) at an initialinstant, production of a model of a dental arch of a patient, or“initial reference model”, and segmentation of the initial referencemodel so as to produce, for each tooth, a “tooth model”, and, for atleast one tooth attribute, assignment of a tooth attribute value to eachtooth model; B′) at an updated instant, acquisition of at least oneimage of said arch, or “updated image”, in real acquisition conditions;C′) search, by deformation of the initial reference model, for anupdated reference model and virtual acquisition conditions suitable foran acquisition of an image of the updated reference model, called“reference image”, exhibiting a maximum match with the updated image insaid virtual acquisition conditions.
 7. The method as claimed in claim1, in which, in the step A), the updated reference model is created witha 3D scanner.
 8. The method as claimed in claim 1, in which, in the stepB), the acquisition of the updated image is performed with an imageacquisition apparatus chosen from among a cellphone, a connected camera,a smart watch, a tablet or a personal computer, fixed or portable,comprising an image acquisition system.